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à Relatively little data exist concerning the 
effectiveness of Navy intercultural relaticns (ICR) training 
programs. In addition, much training research in this area is 
characterized by methodological and design inadequacies. The purposes 
of this. study were to design and test a methodological model, and to 
provide an objective assessment of ICR training impact. It was found 
that training had a modest but significant effect upon the attitudes 
of Overseas Duty Training (ODT)/Personnel Exchange Frogram (PEP) and 


- Human Resource Levelopment Center (HRDC) IDR Personnel. ODT/PEP 


perscnnel changed significantly on 13 of 28 scales and HRDC personnel 
on 9 of 24 scales. Scales measured self-actualization, flexibility, 


, tclerance of ambiguity, acceptance of self and others, leadership 


styles, and basic motivational patterns. The failure to detect a 
greater degree of change may have been due to varicus test ceiling 
effects or to the nature of the change process itself. Although the 
real test of program impact necessitates validation against external 
or in-country criteria, the results did indicate that the impact of 
ICR training, although modest, was consistent with the hypotheses 
generated for the evaluation of the training objectives of the 
program. (Author) i a 
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FOREWORD 
i © "o ) - 
^ This research' was performed under Work Request PO 3-00081 (Feed- 
back in Human Response Training). The research vas initiated im response 
to a request from the Human ‘ReSource Development Project Office (HRDPO) 
uttier Chief of Naval Personnel (Pers-P). An interim report has been 
published as Naval Personnel and Training Research Laboratory Research 
Report SRR 73«20, The Effectiveness ‘of Intercultural Relations Training 
for Vietnam Advisors, June 1973.: The purpose of the present research was 
to “determine if intercultural Relations (ICR) training programs were . S " 
having desired effects in terms of attitudinal and long-range behavioral 
changes. “The training programs evaluated' were Overseas Duty Training (0DT)/ 
Personnel Exchange Program (PEP) and Human Resource Development Center (HRDC): `> 
'ICR Specialists training. dp 
i The Assistance of the ICR training staff of the Naval Amphibious School, 
Coronado, California, is gratefully acknowledged. 
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Intercultural relations (ICR) training prograns are designed to neet 
the folloWing objectives: to extend United States diplomacy ovérseas, 
to increase job effectiveness, and to increase tour satisfaction. v 
. The primary purpose of the research described in this report was to 

examine the degree to which such training programs (Overseds Duty 
Training (ODI), Personnel Exchange Program (PEP) training, and Human 
Resource Development Center (HRDC) ICR Specialist Training) were having 
desired effects in terms of attitudinal and EE Change, 


‘Since much trafning research is characterized by methodological and |". 
design inadequacies, an additional purpose was tọ develop and utilize . N 
a methodological approach which employed scientific standards of ex- > 
perimental design. This ‘approach was designed to provide information 
which could be used to improve and strengthen ICR training and to 


provide an objective assessnent of training effectiveness and impact., 


A. RC r 


L ' . 
Following the specification of program objectives, a series of standard- 

* ized measures was selected for assessing relevant attitudinal change, - 
Baséline data were collected from ODT/PEP. and HRDC experinental. group 
trainees on scales of flexibility, self-acceptanee, acceptance of, 
others, 1 adership style, level of self-actualization, toleranca of 
anbiguity, and basic motivational patterns. „Pretest and.posttest CG 
information was used as a reference in assessing skills following : i i 
3 weeks of training for ODT/PEP personnel and 6 weeks of training for x 
HRDC personnel. Tests were also adninisteted to a control group and, 

. 'to measure test reactivity, to other groups of personnel tested only 

' after training. Pretest and posttest difference scores were tested . - 
for statistical significance. It was aüypothssized that ICR training 
would have a greater impact upon attitude chànge than that obtained 
in a comparable cóntrol group. ` Unfortunately, long-term effects were 
not examined since follow-up research procedures were not carried out. 
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"Minimal support was found for the hypothesis that the experiiental 
groups changed significantly more tnan the control group. It was 
found that test cel ing effects, due to the initial level of scores, 
limited the azoumt of echange. The nature of the change process itself 
may account for the modest changes found. The information which 
is provided, hdwever, does indicate that the imbact of ICR training 
is consistent with specified ghort-range'training objectives and goals. 
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D De E : 1 
Follow-up procedures are recomended to val 
against ‘external criteria such as: car 
'objective measures of ‘on-the-job peri 


formance, and attitudes toward 
host nationals. The data bank develo? 


idate change measures 
. e < . P 
eer and job 'satisfaction, , 


ed. for this evaluation is avail- 
| able for these follow-up purposes. "e" 


\ = - . H 
Due to the considerable iten content overlap between the various 
attitüde Scales, it is recommended tHat empirically constructed keys 
be developed to provid 


. e a hotogeneous. measure of cultural awareness. 
This would reduce considerably t 


he tine required for attitude measure- 
mert without reducing predictive efficiency. 
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An. Evaluation of Intercultural Relations Training s, 
for Hany Overseas Personnel 


"E Introduction 


Ji Program Description " s > 
In July 1972, the «Chief of Naval Üpstations ordered all commanding 
officers of overseas shore activities, afloat units,,. ,and. those CONUS units 

whose operations affect' the SChEVEeIES of overseas-based personnel to: 
"initiate and continue action program which affect. positive re- 
lations between commands and' foreign nationals and which assist individual 
Naval personnel.and their families to work effectively, live with dignity 
and satisfaction, and function a positive representat iyena of tHe Navy and 
the: United States while overseas." . , 


3 
1 


“The Ne Relations | Gom — conducted by the Naval 
Amphibious Base at Coronado,, California, and at Little Creek, Virginia 
were designed to meet this objective. 


^g; d i 


These ICR programs utilize various. innovative materials and methods. . 


Training includes the use of classroom exercises such as group discussions, 


role-playing, case studies, and films. The programs are déveloped around 
experiential and cognitive learning in small groups, with active trainee 
participation and’ two-way communitation. This approach is designed to 
create both student involvement and opportunities to practice interpersonal 
skills? “Implicit: in this approach is an appreciation that technical exper- 
tise and, language training are ,necessary, but not sufficient, conditions 
for effective’ in-country behavior and cross-cultural adjustment.” Specifi- 
cally, training deals with perceptions of host nationals, concerns about 
interaction with host nationals, and preparation for cross-cultural inter- 
actions. Major training modules include awareness, cultural systems, 
culture shock, change concepts, problem-solving skills, verbal as well as 
nonverbal communication; arid comparative linguistics.: The rationale be- 
hind training is that differences in values and assumptions are typically 
the basis of conflicts and misunderstanding in: dealing with host nationals. 
Therefore, training concerns the reduction of unrealistic expectations 
through greater knowledge about the new culture, such as information about 
customs, geography, language, and history. 


In order to:provide predeployment training to personnel going over- 
seas, several ICR programs were developed. The Personnel Exchange " 
Program (PEP) was designed to prepare personnel for assignment to à 
foreign Navy (shore-based or on-board ship), and the Overseas Duty Train- 
ing (ODT) program was designed for personnel assigned to an overseas 
U. S. Navy facility. The content of both 3-weex training programs 
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was identical, and results in this report combine both programs. 
Another training program, the Human Resource Development Center (HRDC) 
ICH Specialist training program, was designed to prepare personnel to 
become ICR instructors. This program contained modules in teaching 
techniques, group processes, course preparation, and testing and 
evaluation procedures. 7 i 


2, Review of the Literature 


The major emphasis of ICR training prograns has been either on Che 
selection of personnel to be trained or on ‘the development and institutional- 
ization of programs (Lau, 1974). Relatively little data exist concerning 
program evaluation and measurement of effectiveness (Brislim, 1970; Hoehn, 
1966; Foster and Danielson, 1966; Haines, 1964; and Wight, 1970). The 
same general pictyre characterizes other group-centered training, such 
as, management and organizational developnent training (Hand and Slocum, 
1972; Lieberman, Yalom, and Miles, 1972; Campbell and Dunnette, 1968; 
and Miles, 1964). In addition to a dearth of evaluative data, major 
methodological and design inadequacies have not been overcome (Campbell 
and Dunnette, 1968). The vast majority of training evaluations have 
depended upon self-report techniques. Objective measures of behavioral 
and attitudinal change have rarely been develoded or utilized as criteria . 
of program effectiveness. Specifically, the major methodological flaws 
in the evaluation of group-centered training prograus include: (a) a 
lack of adequate base-rate or pretraining measures, (b) a failure to 
include matched control groups, (c) a lack of independent observers 
to rate attitudinal or behavioral change, (d) a failure to control for 
the effect of pretest measures On posttest reasures, i.e., test reactivity, 
(e) a failure to; employ .dependert measures consistent with group goals, 
and (f) with few exceptions, failure to include longitudinal follow-up 
as well as transfer of training measurenent. In addition, the great 
majority of training research has-been concerned with internal change 
criteria such as attitude and opinion shifts related to what trainees, 
thought they had learned. In order to effectively evaluate training, 
Andrews (1966), Campbell et al (1970), and MacKinney (1957) have indicated 
a need for the utilization of control groups and relevant pre and post 
neasures. Regarding test reactivity, Fishbein and Ajzen (1962) make 
the point that change or difference scores tend to lead to invalid 
conclusions unless post-only designs are utilized. The few studies that 
have used a scíentific- design tend to snow that group-centered training 
nas a positive effect on attitudes and perforzance. However, only 
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luhile the effectiveness of the 6-week HREOC program is evaluated in thiy 
study, the progran was revised in February 1974 aud conclusions reached 
may not generalize ta the revised program. 
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à z - x o , 
five group-centered training programs, utilizing before and after measures 
with experimental and control proupa, were found in the literaturę by ^ 
Hand and Slocum (1972). ç 

H F 

esri tE the lack of evidence concerning the effect of ICR training 
and the often technically inadequate nature.of the research reviewed, an 
overall impression emerges that it is, possible ‘for training to have a 
positive impact on attitudes and effectiveness in the foreign setting. 

It is clear that personnel who are sent overseas with no preparation for 
the culture-related aspects of their jobs, except the reading of some hand- 
books or discussions with people who have served in similar assignments, 
often have been found to perform in an unsatisfactory manner: (Fiedler, 
Mitchell, and Triandis, 1971). d 


The immediate objective of the Navy's ICR training programs is to- 
change the attitudes and beliefs of trainees in a positive direction. 
However, it is necessary to show that a relationship exists between sucn 
attitudes and beliefs, and actual behavior. Ajzen and Fishbein (1973) 
have shown that there is a consistent relationship between attitudes, 


"beliefs, and behavior. It should be noted, however, that this relation- 


ship has been shown to be unstable over.time. The longer the interval 
between measurement of attitudes and beliefs and the measurement of the 
behavioral SES the less stable the relationship. 


In an a&ritar study on the effectiveness of E ICR training 
for Vietnam advisors (Lau and Curtis, 1973), the conclusion was reached 
that the program was partially effective in terms of the attitudinal 
changes of trainees as compared to changes in a comparable control group. 
Due to curtailment of the program, however, the study utilized a rela-' 
tively small number of subjects and, for the same reason, no posttraining 
measures were obtained. ta 

Intercultural relations training often includes role-piaying exer-' 
cises, case studies, group discussions, and the written and oral presenta- 
tion of cognitive information. Although. Wight (1970) considers experien- 
tial learning to be the cornerstone of ICR training, conclusions reached 


by an evaluation of the total program cannot be attributed solely to ex- ~ 


‘periential learning. There have been no conclusive studies that have 
compared the relative effectiveness of training without experiential learn- 
ing to training with experiential learning. With reference to the ICR 
evaluation described in this report, it is difficult to isolate the module 
or modules that contributed most to changes. The total programs are 
evaluated, but the contribution of each module is not. à 


An area that needs considerable attention is the identification of 
critical behaviors which constitute effective overseas performance. With- 
out this information, it is difficult to assess the success of training. 


Research on the effectiveness of ICR training has been highly dependent, 
on usage of verbal, self-report measures, and, with tne Ku diese of a 


report by Yellen and Hoover (1973), little data kava bs zenerated on 


actual behaviors toward host nationals. ` p 
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3. The Present Effort 


The purpose of this research was to determine’ whether or not opt /PEP 
and HRDC training was having the desired effects in'teras of attitudinal 
and long-range behavioral change. The research design was originally 
organized according to two types of evaluative criteria--internal (or 
process) such as attitudes and external (or product) criteria, including 
longitudinal, in-country performance measures. Since it is conceivable 
that internal criteria were achieved and external criteria were not, 
measurement of the latter would indicate whether skills learned in ICR. 
training are transferred and practiced when graduates are stationed in ` 
the foreign setting. Several research procedures and instruments were 
proposed for the follow-up phase (Lau and Blanchard, 1973). These were 
as follows: l e . 

n 

a. Course Critique. This measure is sinilar to.the current in-house 
course critique administered at the conclusion of ICR training. It was 
designed to obtain information regarding how ‘personnel perceived the 
impact of training 6 months after graduation. Relevant considerations 
concerned such elements as reactions toward training, how training could 
be improved, how training had helped the graduates, and what effects 
training had upon self-reports of job performance and job satisfaction. 

5 E á à E 

b. Readministration of Change Scales. It is not known whether sig- 
nificant attitudinal ‘change persists over time, or becomes modified by 
in-country experiences. The initial design of the evaluation included 
readministration of several of the change scales. 


- 0 

c. In-country Attitude Survey. A critical criterion of program 
effectiveness is the actual behaviors and attitudes of graduates now 
in-country. One relevant question concerns whether. graduates have more 
favorable attitudes toward host nationals than personnel who have not 
been exposed to ICR training. A survey for ODT/PEP graduates dealing 
with these attitudes was developed aad is included in this report as 
Appendix A. 


Largely due to inadequate project funding, these follow-up procedures 
were not utilized in this evaluation. As a result, in-country performance 
and the stability of change were not measured. It should be emphasized , 
however, that pretest and posttest attitude test scores are available 
for follow-up purposes.’ The authors encourage Navy progran managers. 
to consider making an attempt to measure the long-tern effectiveness 
of ICR training in terms of external criteria. | 


Although proposed :follow-up procedures did not materialize, this 
report presents useful feedback information regarding the attitudinal 
changes that occured during the 3-week ODT/PEZ2 and the -week HEDC ICR 
specialist training P ograms. 
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B. Procedures 


* 
l. Specification of Attitudinal Goats l : : 
the initial step involved a description of progran background 

and an examination of training ‘goals. Discussion with trainers 

and program managers, results reported in the psychological litera- 
ture for similar training programs, and an earlier stugy on the 
effectiveness of ICR training for Vietnam advisors (Lau and Curtis, 
1973) indicated that the following were major adjustment ot 


attitudinal training goals: S 


a. Increased ability to tolerate ambiguity 


b: Increased adaptability 


H 


c. Increased self-awareness.and self-insight concerning one's 
behavior and its effect upon. others 


d. Increased interpersonal sensitivity (empathy) 
e. Increased self-acceptance and acceptance of others 


f. Increased consideration shown to co-workers and subordinates ^. 
and maintenance of a high task orientation 


g. The development of an attitude of openness to new experiences 
h. Reduced dogmatism and ethnocentrism. 
2. Selection and Development of: Relevant Tests 


This step involved the selection and/or construction of objective 


> evaluative instruments that measured attitudes specified as being 


training goals. The research.literature indicated that a number of 
published tests showed promise for measuring these’ changes. The 
instruments chosen have been found by other researchers to measure 
the effects of human relations and management training in the indus- 
trial setting with àcceptable levels of reliability and validity. | 


The £ollowing instruments were utilized to assess changes resulting ' 
from PEP/ODT and HRDC training: 






e NE í 42 
2 : : S : : i : i 
The Peer-Nomination Form used in the Vietnan ICR evaluation was not 
used in the present study. This was because class size rarely was large 


enough for this form to be useful. 
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"(Shostrom, 1966). Scores are provided on time competence, inner directed, 
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a. Leadership Opinion Questionnaire (L00). This questionnaire 
measures two independent dimensions of leadersnip style--structure 
and consideration. The LOQ has been used in evaluating a variety of 
mapagement development programs (Fleishman and Harris, 1962; Fleisiuan, 
1969). 








High scores on consideration characterize leaders who allow subor- P A 
'dinates more participation in decisicn-making and two-way communications. 
High scores on structure characterize leaders who organize and define 
group activities toward goal attainment and define roles that the leader 
expects each subordinate to assume. 

e 


b. Flexibility Scale (F). This instrument measures a variable 
hypothesized to be associated with resistance to attitude change. It 
represents an experímental instrument desigued to identify individuals 
likely to have problems in accepting criticisn, adapting to new situa- 
tions, and/or in accepting the values of other individuals. 


c. Survey of Interpersonal Values (SIV). This is a measure of basic 
motivational patterns (Gordon, 1960). Scores are provided on need for m 
recognition, independence, leadership, benevolence, conformity, and ` 
support. ` ; 


d. Self-acceptance (SA) /Acceptance of Others (AO). This instrument , , 
was adapted from two scales originally developed by Berger (1952). C 


Self-acceptance is defined as the extent to which an individual is guided ' 
by internalized values (rather than external pressure), a sense of self- 


worth, and an absence of self-consciousness. The acceptance of others 


scale measures the degree to which an individual perceives others without 
preconceptions and refrains from placing his values on others. 


e. Tolerance of Anbiguity (TA). This instrunent is designed to 


measure the. position of an individual on a continuum from a strong 
tendency to perceive ambiguous situations as threatening to a strong in. 
tendency to view ambiguous situations as desirable (Budner, 1962). 


f. Personal Orientation Inventory (POI). This inventory provides a 


measure of Maslow's concept of self-actualization as it relates to.per-- 
sonál development and the ability to develop interpersonal relationships 





self-actualization value, existentiality, feeling reactivity, y, spontaneity, m 
self-regard, self-acceptance, nature of man; synergy, aecentanca of . Ü 
aggression, capacity for intimate contact. Hd 


£. Change Questionnaire (CQ). This questionnaire was used to 
measure self-reported change resulting from participation in training. 
It was admin‘stered only at the conclusion orf training, at which time 
trainees ware asked to indicate the direction of change (if any) by 
checking a series of 25 bipolar adjectives (e.2., tense--relaxed, 





rigid in thinking— flexible in thinking, etc.). te the trainee. had not 
changed, he was instructed to'leave the item blank. 
lo 


c» 








h. Biographical Questionnaire (80). This instrument asked questions 
2 of trainees such as age, educational background, pay grade, and number 
— ‘of enlistments. -This information was gathered in order to assess the 
influence of biographical characteristics upon the change measures. 


t 


3 Colisctiox of nara from Experimental and Control Groups 


» D N D 
With the exception of the Change Questiounaire, baseline data were 
collected from trainees on all of the above neasures. This infornation 
was used to assess change following training. 


? 
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In order to insure that changes in the experimental group were | 
not due to mere passage of time or low test reliability, tests were 
also administered to a control group. The control group in this 
study was tested during language training.at the Defense Language 
Institute, Monterey. Language training precedes PEP training. The 
pretest to posttest interval in the control group was 4 weeks, which 
'was reasonably equivalent to the ICR training intervals of 3 or 6 weeks. 


To measure the effect of pretesting on final testing, a sample of 
trainees in both ICR programs was tested only after training (post-only). 
This was done to determine the degree to which trainees night have been 
sensitized by taking the pretests. - i 


D 
Li 


4. Assessment of Training Effectiveness 


This step involved an examination of attitudinal change experienced 
by trainees. Pretest and posttest scores on the various scales were 
compared and tested for statistical significance, using "t" tests 
(McNemar, 1960). This procedure was followed for both the control and 
experimental groups. It was hypothesized that ICR training would have 
a significant impact upon attitude change, whereas no significant attitude 
change would be found in the control group. To measure test reactivity, 
scores for trainees im the post-only groups were compared to posttest 
scores of trainees in the experimental groups. In order to get a clearer 
picture of the nature of the attitude changes that occurred Ouer training, 
a factor analysis of the intercorrelations among /the various tests was 
conducted. : Wë 
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{here were 116 trainees in the 12 ur and four PEP. classes tested "^ 
fron December 1972 through April 1974. Uf this uumber, 16 were 
dependents. In the six HRDC classes tested from November 1972 through ` 


February 1974, there were 46 trainees, of whom two were dependents. 
In the post-only groups, 19 ODT/PEP and 13 HƏC graduates were tested. 


Pretests and posttests were also administered to 20 trainees 
enrolled in langyage training at the Defense Languagé Institute. 

As noted earlier, these trainees were scheduled to begin PEP: training 
after completion of language preparation. , l 
Table 1 shows the pay grade, educational level, age distribution, 

number of overseas deployments, and other Selected biographical «X 
characteristics for the experimentad and control groups. Responses 

for the two post-only groups are not presented. There were no sub- 
stantial differences on biographical data characteristics between 
experinental and the respective ODT/PEP or HRDC post-enly groups, Or 
between the control group and the ODi/PEP group.  HRDC trainees differad 
from ODT/PEP trainees and the control group in that, the former 
'consisted of more officers with, a-consequent lower age and higher level 
of education. "To some degree, ODT/PEP trainees reported &# more 
favorable attitude toward Navy life, and a nigher percentage planned 

to reenlist or extend on active 'duty. ` l 
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TABLE 1 


Characteristics of Trainees in the "sa, 
Experimental and Control Groups l) K 








DN e" 
. ODT/PEP . HRDC ^ 
Experimental Group| Experimental Group Control Group 
(N-116) ., (N=46). - (N=20) : 
Item . N % N Z. 
l. Present Pay Grade : pe 
a. E~5 or belów | 17 15 8 A 17 0 0 
b. E-6° : 14 = 12 6 (^u3 ^4 5 25 
c. E-7 to E-9. ! 38 32 B f pil 7 35 
d. Officer | 31 27 22 j / 49 9 25 
e. Dependent .16 * 1P .2 A 3 15 
2. Educational Level 
a, Not High School dt b d 
Graduate Z 2 i 2 1 5 
b. High School Grad- j ` i 
uate (or GED) 50 i 42 . 4 9 [12 60 
c. Some College '23 20 15 :33 2 10 
d. College Graduate 18 16 19 41 2 "10 
e. Post-Graduate uL 
' Work 23 20 T 15 3 15 
3. Age 
U b 
a. Under 25 Years 11 9 ' 26 56 1 5 
b. 25-29 |: | 29 25° 11 24 2 10 
c. 30-34 7 43 37 3 7 8 40 o 
d.. 35-39 |. 23 20 5 11 6 30 
e. 40-49 . 10 9" EE 2 3 15 
B d 7 : 
| 4, First Enlistment] š 
»a. Yes i 12 12 12 27 1 6 
b. No 88 88 32 73 16 94 
5. Plan to Reenlist or l 
Extend on Active Dutyl 
a. Yes | n 71 20 45  |14 82 
b. No ; 20 ' 20 11 25 2 12 
. c, Undecided QE. TOM 9 13 30 1 6 
(Continued) 


NOTE: Leon these items dependents are not included. 
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6. Attitude toward 


the Navy+ 
a. Very satisfied 57 
jb. Satigfiéd , 738 


c. Neither, satis- 
fied nor dis- 
satisfied 5 
d. Dissatisfied 0 
e. Very dissatisfied 0 


7. Number of Overseas 


Deployments (4 months 


or more) 

a. None il 
b. One œ ll 
c. Two 13 
d. Three - 8 


Four or more 


NOTE: lFor these items dependents are not included. E 





ODT/PEP ` 


E erimental, Group 
e (N=116) 















HRDC | 


Experimental Group 
Ñ (N=46) 









Control Group 
(N=Z0 ) 
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| } D. .Results and Discussion 8 g 
) U : 3 * e , * 
Á This section is organized around several research questions. First, 


(~~ significant changes on the various attitude scales from the pretest 
to the posttest are assessed in relation to whether or not changes 
are significantly"larger for trainees in the ODT/PEP and HRDC. 
~ experimental groups than the control group. . Sécondly, the information 
" on self-reported chhnge is examined. Thé next question investigated 
coficerns the consistency and nature of what is measured by .the various, 
tests. A final question examines the pretest differences between ODT/PEP 
x ^ 'and HRDC trainees. This analysis sheds some light on the’ selection 
process for ICR-trained personnel. : . ? 


D 





PC 


R . l.. Test-measuréd Attitude Change CMM 
08 ' — Various hypotheses were generated regarding change às a result of 
ICR training. . These were Gees upon results reported in the literature 
for selected scales,and upon p evious research on the ECR Vietnam program. 
Hypotheses were'also based upgn specified course goals. Specifically, 


it was hypothesized that significant positive change would be found on - 
the following: consideration, structure, independence, self-acceptance, 
acceptance’ of others, tólerance of ambiguity, flexibility, and six of 

12 POI ‘scales (extentiality, feeling reactivity, spontaneity, self- b 
regard, self-acceptance, and capacity for intimate contact). 


.. The first analysis centered around the question of pretest differences 

between the ODT/PEP experiméntal group and the control group>. It was 

found that there was significant differenct on three of 24 scales 

(P < .05). { The control group scored lower than the experimental group on 
* tolerance of ambiguity and support, and significantly higher on con- 

formity. Despite these differences, it was concluded that the two groups 

were essentially the same.before training. D ' 


.. The second analysis concerned the effect of pretesting on final test- 
ing. In this analysis, the differences between post-only groups and 
posttest scores for trainees in the two experimental groups were compared. 
It was found that ODT/PEP trainees in the post-only groups scored signi- : 
ficantly lower on two POL scales--inner-directed and feelings reactivity 
(p:< .05). The HRDC post-only sample scored significantly lower on 

', flexibility, independence, and three POI scales (inner directed, féeling 


.reactivity, and capacity for intimate contact). With respect to the POI, 
there is some evidence for test reactivity. 
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NH the pretest, it was not anticipated that HRDC trainees would be com- 
parable to either. ODT/PEP trainees or the control group which consisted 
of prospective PEP trainees. Thus, these differences were not tested. * 
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Although test reactivity was not found in the ICR Vietnam evaluation 
and sample sizes in the present study were small, exposure to the pretest 
appears to lead to higher, scores on several POI scales following training. 


Table 2 presents the means, standard deviations, and critical ratios 
between pretest and posttest scores for the two experimenta} groups 
(ODT/PEP and HRÜC) and the control group. Trainees in ODTfÍPEP earned 
significantly different scores on 13 of 24 change scales (p < .05), HRDC 
trainees changed significantly on nine of 24 scales, and the control 
group changed significantly on only two--independence and capacity for 
intimate contact.  Reasonably parallel attitude changes were found in the 
two experimental groups. Both groups changed significantly on structure, 


independence, inner: directed, feeling reactivity, and the POI self-accep- 
tance scale. . f ` 


When the absolute number of experimental group changes is compared 
to control group changes, results indicate that training had a modest 
but significant impact' upon attitudes. The direction of these changes 
largely supported the liypotheses generated for this evaluation. E 


It should be noted that the initial level of scores is an important 
consideration in studies resulting from training. If pretest scores 

are already exceptionally high, it is unreasonable, due to a ceiling 
effect, to anticipate marked change over training. ‘For ODT/PEP trainees, 
Scores on consideration were considerably higher (approximately one 
standard deviation) than those reported'in the LOQ Manual (1969) for 

Navy Officer candidates. 'HRDC pretest scores on consideration WEES HYS. 


higher. Norms reported by Berger (1952) for the self-acceptance scale 


show that pretest scores for both experimental groups were also approxi- 
mately one standard deviation higher than scores of a sample of college 
students. Norms for acceptance of others are not reported. Pretest 
scores on independence for ODT/PEP reached the 59th percentile and the 
7lst percentile for HRDC trainees. Finally, college student norms in 
the POI Manual (1966) indicate that ODI/PEP pretest scores on POI scales 
ranged from the 74th percentile for feeling reactivity to the 88th 
percentile for inner-directed. These findings indicate that ODT/PEP 
trainees represent a highly select group before training. For this 
reason, substantial pretest-posttest change should not be expected on 
many of these scales. ; , E 


Despite these ceiling effects, trainees in the ODT/PEP group earned 
significantly higher posttest scores on flexibility, independence, and 
self-acceptance, and significantly lower posttest scores on Structure 
and needs for recognition. Increases in POI scores, although 


eg 


re should also be noted that the maximum score on the experimental 
flexibility scale is 48. Since HRDC pretest scores were 42.5, significant 
change on this scale should also not be expected. 
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modest, reached significance for eight of 12 scales, including inner 
directed (an overall measure of self-actualization and growth in inter- 
personal interaction) and several. subscales, each of Which measures a 
conceptually important element of self-actualizatior. As hypothesized, 
significant change was found on existentiality (greater flexibility in 
the application of values), feeling reactivity (sensitivity to needs: and 
feelings), capacity for intimate contact (a measure of the ability to 
develop ueantusFul relationships and perceive situations from another's 
position), spontaneity (the ability to express feelings), and self- 
acceptance (acceptance in spite of weaknesses or deficiencies). Although 
significant increases.were predicted in self-regard, consideration, struc- 





ture, and tolerance for ambiguity, these hypotheses were not supported. 
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Trainees in the HRDC experimental group earned significantly higher s 
.posttest scores on support (needs to be treated with understanding), 
indgpendence (needs to be free to make decidions), and acceptance of 
others (the ability to accept other individuals with different valües). 
Significantly lower scores were found on structure, conformity (doing 
what is accepted and proper), and benevolence. Increases reached signi- 
ficance for three of 12 POI scales inner directed, feeling reactivity, 


and self-acceptance. 8 . 





To a great degree, the impact of ICR training is consistent both with 
respect to the dimensions of the Profile of.Cross-cultural Readiness . 
(PCCR) presented in the Handbook for Overseas Diplomacy (1973) and the 
training objectives developed for this evaluation. 'For example, one dimen-. 
sion of the PCCR refers to self-awareness skills and another to acceptance * s 
level. The change scales used in this evaluation, particularly self- 

„acceptance and acceptance of others, appear to measure these two PCCR 
, dimensions. Cbanges on these scales indicated that trainees increased in 


"the extent to which they are guided by internalized values, a sense of. 


P. 
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self-worth, and the acceptance of others whọ, may live by different values s, 
A second PCCR dimension, adaptability, appears to be closely related to 
the flexibility scale used in this evaluation. Due to ceiling effects 
noted above, the'HRDC experimental group did not change on this scale. 
However, higher scores in the ODT/PEP group indicate an increase in the 
ability to be flexible in forming attitudes, in accepting the values of 
other individuals, and a readiness to make changes in behavior. Finally, . 1 
capacity for intimate contact seems to overlap with another significant e 
PCCR skill--empathy.. It was found that ODT/PEP trainees significantly 
increased their ability to see situations from the perspective of other 
individuals. Uo a 


It was found that after training both experimental groups had signifi- 
cantly lower scores on structure. This decrease reflects less concern © 
for defining and structuring the subordinate's role toward goal attainment. 
No changes were observed on consideration (i.e., the input of subordinate 
influence on policy decisions). To some degree, the decrease in structure 
is consistent with signifitant increases in needs for independence 
in both experimental groups and decreased conformity scores in the HRDC 
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group. Carron (1964) reported the same findings regarding decreased 
structure for supervisors receiving management training in a chemicals: 
company.: He found that more emphasis on planning and organizing revetsed 
this decrease and, in subsequent training cycles,” supervisors changed | 
toward a high consideration-high- structure pattern. Since high scores 
on both scales of leadersh$p style are likely to maximize à variety of 
different effectiveness criteria, including job performance, attitudes, 
and proficiency ratings, the results réported for the ICR programs are 
nct interpreted to indicate a favorable course outcome. Examination 
of the content of the‘programs might indicate that more emphasis on 
planning and organizing was needed and could fesult in both higher. 
tonsideration and structure scores following training. . 


The critical test of program effectiveness is a direct statistical 


test of the question: Did the HRDC and ODT/PEP experimental groups change ' 


significantly more on any of the attitude measures than the control group? 
It was found that ODT/PEP trainees decreased significantly less on 
consideration that tha control group (t = 2.14, p < .05), and decreased 
significantly more on trucfure than the control group (t = 2.60, 

< 05). On the acceptance of others scale; ODT/PEP trainees tended to 


a en D 
s incréase significantly more than the control group, whose.scores decreased 


(t = 1:97, p «.06). HRDC trainees decreased significantly more than 
the control group on structure (t = 2.45, p < .05) and increased mre 
on acceptance of others than the control group (t = 2.84, p < .01). 
No other differences reached significance. > 
These results provide minimal. support eor the hypothesis that the R 
experimental groups changed significantly more than the control group 
As noted earlier, ceiling effects on self-acceptance, the LOQ, and other 
scales, including the POI, may account for the modest changes found. 
However, changes were generally in the hypothesized direction. A greater 
number of significant changes wére found n the experimental groups than 
in the control group. 


Due to the considerable overlap between the various scales. (and 
particularly the items that make up each of the scales), the develop- 
ment of an empirically constructed key designed to provide a homogeneous 
measure of cultural awareness is indicated. Such a key could lead to 
shorter, more effective change measures. 


The inability of the scales to identify significant change may be 
the result of the change processes themselves. One could argue whether 
3 weeks of training can drastically change rélatively stable individual 
characteristics, such.as self-acceptance, interpersonal sensitivity, 
and flexibility. What may be occurring over training is the development 
of an attitude of. receptivity: toward change, This receptivity may mani- 
fest itself in the form of long-range attitudinal and behavioral effects 


only when the graduate has the .opportunity to practice and receive feedback 


on these new skills, i.e., in a foreign settzfíg. Obviously, it is more 
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important to examine change over a substantial ‘period of time than change 


immediately after the conclusion of training. 


2. Sel f-Reported Change l 


The change questionnaire, a measure of self-reported change, was 
administered to trainees at the conclusion of training. Table 3 shows 
the number and.percent of trainees in the experimental and control groups 


reSponding positively to Change Questionnaire items. Both ODI/PEP and” 


HRDC trainees reported the same kinds of changes. : For example, both 
groups reported the largest positive change on understanding of others 
(item 16), flexibility in thinking (item 24), and sympathetic listener 
(item 25). Se a S . 


For most items, a greater percentage of the experimental groups 
reported positive change than the control group. The average nunber 
of positive changes in the ODT/PEP group was 10.3. In the HRDC group, 
the average was 11.1. The control group reported an average of 5.8 
changes. The difference between the number of changes reported by the 
ODT/PEP group and the control group was significant (t = 2.44,,p < .05), 
as wàs the difference between the HRDC group and the control group .(t 
= 2.25, p €a405). ` ` l E 


With reference to self-reported change, it is concluded that ICR 
training had significant impact. Both experimental groups reported more 
self-perceiyed change than the control groüp., In general, this impact 
was compatible with program góals and objectives. 


3. Factor Analysis of ODT/PEP Test Scores 


The correlation matrix presented for ODT/PEP in Table 4 shows the 
pretest correlations between the attitude tests, and provides a measure 
of the construct validity of the various scales. As can be seen, the 


inner directed scale fs closely related to the other POL scales. This 


is largely because of item overlap between this scale and the various 
other POI scales. This overlap suggests that the length of this scale 
could be considerably reduced without reducing content coverage. The 


independence of the two LOQ scales of consideration and structure is 


confirmed. Flexibility is seen to be moderately related to tolerance ` 


of ambiguity and spontaneity and, as might be expected, negatively related 


to conformity. Further, the self-acceptance scale of Berger shows con- 
struct validity by its high relationship .to the POI scales of self-regard 


and seif-acceptance. 


Due to the number of moderate. correlations between the attitude tests, 
an orthogonal factor analysis,was performed on pretest and posttest 
scores in order to get a clearer picture of what the various scales 
measure. It was anticipated that this procedure would disclose an under- 
lying pattern of relationships such that the data could be reduced to 
a smaller set of components. 
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2. Dealing with problems l 
3. Trusting in relations with friends 
T 4^ Ab, Relaxed 
g 5. Other-person centered 
6. Enjoy being with others 
7. High aspirations 
8. Easy going 
9. Working well with co-workers 
10. Clear in thinking 
1l. Feel good about self 
| 12. Working well with people in 
authority 
13. Sincere 
l j 14. Liberal ` 
15. Decisive 
16. Understanding of others 
17. Self-control 
18. Self-assured 
19. Able to help otliers with problems 
20. Energetic 
21. Optimistic toward future 
22. Independent 
23. Unshakabie 
24, Flexible in thinking 
25. Sympathetic lictener 
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"TABLE 3- 


N 


1. Trusting in relations with co-workers 41 


65 
43 
46 


53 


“47 


41 


EI 


41 - 


49 
AL 
35 
46 
19 
28 
81 


50 


17 


self-reported Positive Change for Experine 






ODT/PEP 


Experinental Group 


(37115) 
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ntal and Control Groups 
















HRDC 
Experimental Group Control Grous - 
(N=45) (N=19) 
yd R 
25 56 ' 16 
28 62 47 
22 49 21 
25 56 1 
17 38 11 
17 38 42 
19 42; 37 
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24 3 2l 
22 49 16 
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14 d 31 1l 
23 ai 32° 
12 MET 2 1l 
10 å ° 22 3 16 
35 78 6 « 32 
15 33 8 Ai 
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Variables loading on to a.factor at .40 or better were considered to 
be major components of that factor. As seen in Table 5, the seven scales 
loàding on pretest Factor l also all loaded on posttest Factor i, with the 
exception of feeling reactivity. All the tests making up Factor l repre- 
sented various POI scales. Of the 12 POI scales, nine are included in 
Factor Structure 1 for the posttests. The additional POI scales of self- 
regard, time competence, and self-actualization value, along with self- 
acceptance and independence, loaded on Factor 1 for the posttests. Since 
independence and self-acceptance are consistent with this theoretical con- 
struct, Factor 1 has been interpreted to represent self-actualization. 


The following results concern posttest factors. Factor 2 shows that 
flexibility and consideration load positively, while tolerance of ambiguity 
and conformity load negatively. Since the tolerance of ambiguity scale 
shows increased tolerance as scores decrease, this scale actually loads in 
the same direction as flexibility and consideration. This factor may best 
be seen as a measure of adaptability. Factor 3 is seen as representing 
acceptance of others. It consists of positive loadings on acceptance of 


` others, self-regard, self-acceptance, benevolence, and consideration; 


and negative loadings on independence. Factor 4 is interpreted to repre- 
sent an awareness factor. It consists of positive loadings on synergy, 
nature of man, and self-actualization value. ` 


As seen in Table 5, factor structures are modified over training. 
Results from this analysis revealed four major posttraining factors of 
seif-actualization, acceptance of others, adaptability, and awareness. / 
This interpretation makes conceptual sense and is consistent with stated 
training objectives. By providing an empirical basis for selection and/or 
construction of relevant measurement techniques, this factor structure 
appears to be of value for future ICR training evaluations. This con- 
clüsion is consistent with an earlier recommendation to reduce the total 
number of scaies through item analysis in order to arrivo at a homogeneous 
measure of cultural awareness. ` 


4. Comparison of ODT/PEP and HRDC Personnel 


It was noted earlier in this report that the two experimental groups 
differed on both biographical data characteristics and pretest scores. 
These pretest differences were tested for statistical significance. It 
was found that HRDC trainees earned significantly higher pretest scores 
on flexibility, consideration, and independence, and significantly lower 
séores on structure and conformity (p < .01). The HRDC experimental group 
also scored significantly higher on three POI scales--inner directed, 
existentiality, and capacity for intimate contact (p < .05). There were 
no significant differences between the two ICR training groups on 


tolerance of ambiguity, acceptance of others, or self-acceptance. 


To a large degree, the selection process for HRDC ICR specialists 
resulted in trainees whose attitudes and psychological makeup were 
i e 
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consistent with stated program objectives suca as adaptability, 


, increased interpersonal sensitivity, consideration shown to co-workers, 


and openness to new experiences. It is not known, however, to what 
degree these characteristics are related to on-the-job performance 
or job satisfaction. ' 


E. Summary and Conclusions 


When the change experienced by ODT/PEP and HRDC ICR Specialists 
was compared to change scores obtained by a control group, minimal 
support was provided for the hypothesis that ICR training would have 
a significant impact upon attitude change. This ray have resulted 
from various test ceiling effects, whereby change was limited by ini- 
tially high pretest scores or from the nature of the change process 
itself.  It.was hypothesized that significant attitudinal change may, 
occur only after the development of receptivity toward change or after 
trainees have had thé opportunity to practice and receive feedback 


'on'newly acquired adjustment skills. Duc to overlap between the 


various tests and the items that make up each of the scales, it was 
hypothesized that devek*opment of an empirically derived key providing 
a shortet overall measure of cultural awareness would provide more 
sensitive change measures. Results from a factor analysis of pretests 
and posttests supported the last hypothesis., 


The real test of program impact, however, concern$ the development 
of externally based, or product, criteria of eifectiveness. This 
involves validation against product criteria such as career and job 
satisfaction and both survey-based and unobtrusive behavioral indices 
of overseas diplomacy. Process criteria—such as relevant standardized 
self-report tests of the type used in tnis evaluation and other process 
indices such as unobtrusive in-class behavioral measures; independent 
observer, peer,.and instructor evaluativis; auu performance on tests 
measuring cognitive or knowledge skills—should be validated against 
these product criteria. This would provide a sore comprehensive 
evaluation of program effectiveness aad indicate whether skills learned. 
in training were transferred and practiced in-country. The follow-up 
instruments and procedures proposed for this evaluation, which included 
administration of course critiques, readninistration of relevant change 
scales to measure the stability of change, and. administration of 
attitude surveys to trained and untrained in-country personnel, would 
have provided a more comprehensive feedback locp. Tho information 
which is provided, however, does indicate that the impact of ICR 
training, although modest, is consistent with the training goals of 
the program. 
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APPENDIX A 


SURVEY OF IN-COUNTRY ATTITUDES AND EXPERIENCES 


l. If you received training, did this training help you to'adjust to 
local conditions?, 


a. 
b. 


Yes, quite a bit 
Yes, a little 


Not very much 


No, not at all 


I haven't received training 


2.- How much more information or training in the following areas do you 
10. feel would have been of benefit to you for your assignment in this 
country? 


3. (a) 
4. (a) 
5. (a) 
B. (a) 
7. (a) 
8. (a) 
9, (a) 
10. (a) 





(b) 


(c) 


(c) 
(c) 
(c) 
D 
(c) 


(c) 


(e) 


Le) 


(c) 


(d) 


.Understanding of prejudice 


a. Training or information was not necessary 
in this area 

b. Training or information was adequate 

c. Some additional training or information 
was needed N 

d. A great deal of additional training or 
information was needed S 


Local customs and courtesies 

Local laws 

History and political background of this country 
Interpersonal communication Bëbee | 
Effects of ee prejudice 

Effects of culture on the way people behave 

Host country military decorum 


System of social status in host country 
s among host country 


nationals 


ll. In general, how are you treated by the majority of host country 
* 


nationals? 


a. 
b." 
Ce 


Treated well 
Treated with indifference 


Treated poorly 
d. Too hard to evaluate 
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12. 


13.- 
20. 


13. 


14. 


22. 


e 
< 


. How would yoü describe your relationship with host-country nationals? 


a. Have geet many solid E 

b. Æ@ccasionally socialize with them and have nade a few friendships 
c. f Have made some casual acquaintances 

d. Rarely associate with them’ D 

e. Avoid association with them . : 

E; 


Have had no opportunity to interact with then 
` H 


What is your attitude about' each of the following aspects of the 
host. country? ; ` 





at G 
St ` Gem T H 

c. Dislike 

d. No chance to evaluate this 
(a) (b) (c) (d) Local customs and courtesies 
(a) (b) (ey: (d) Local shops and stores | 
(a) (b) (c) (d) l Public transportation N 
(a) (b) (c): (d) ` Social system 
a) (b) (c) (d) Military command procedure 
(ay (b) (c) (d) Host-country ‘nationals you work with 
(a) (b) (c) (d) Host-country police 
(a) (b) (e) (d) ° Host-country civilians | 


"What kind of housing are you presently living “iu? 


a. On base 

b. Off base, U. S. military housing 
c. Off hase, host-country housing 
d. Off base, temporary housing 

e. Other than above 


Would you recommend traveling to thís country to a friend? 


a, Yes 
b. No 


c. Don't know 
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23. 


24. 


25. 


26. 


27. 


28. 


29. 


Would you like to return to this country as a tourist? 

a. Yes B bw 
b. No l 

c. Don't know l i 


How long have you been stationed in this country? 


0-3 months 
b. 3-6 months s 
c. 6-9 months 


d. 9-12 months 
e. More than 12 months 


H 


How would you describe your ability to speak the host-country language? 


a. A few words to no words 

b. Simple phrases 

c. Halting conversation. 

d. Speak with ease 

: e 

Would you like to be stationed or home-ported in this country again? 
- e ` Y » ^ 

a. Would like to i E 

b. Indifferent ;. 

c. Would dislike (it 


Have your views about relationships with foreign nationals changed 
since you left the States? 


a. Much more accepting 
b. A bit more accepting 
c. About the same 
d. A little less accepting 
e. Much less accepting 
Ñ 
Have you had any unpleasant experiences with host-country nationals 
since your àrrival? l 3 


a. Yes, quite a few 

b. Yes, some 

C. Only one or two isolated insiances : 
d. None at all 


Please give examples, if you answered a, b, or c, above. 


Soyo 
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3l., 


32.- 
42. 


32. 

33. 

34. 

: 35. 
36. 

| 37. 


38. 


39. 


s ` 
When interacting with host-country nationals, do you generally feel--- 


a. Very uncomfortable or nervous? 


ps d Slightly üncomfortable or nervous? 
c 


No noticeable unpleasant feelings? 


What is your opinion on the overall quality of instruction you 
received in preparation for your overseas assignment? 
Ñ U 
a.. Superior 
b. Above average 
c. Average 
d. elow average 
e. Poor 
f. Did not receive any training or instruction 


While you are stationed in this country have you, or do you intend 
to do the following? ' 


a. . Have already 
b. Strongly intend to 
c. Probably will 
d. Probably will not 
Se Plan to avoid 


(a) (b) (č) (d) (e). Visit places of historical importance 





(a) (b) (c) (Cd) (e Become close friends with several host-nationals 
(a) (b) (c) (da) (e) Visit local taverns 

(a) (b) (c) (d) (e) Attend sporting events 

(a) | (b) (c) (d) (e) Spend most of your Exod time with Americans 


(a) (b) (c) (d)'(e) Patronize local food stores 


(a) (b) (c) (d) (e) Influence host nations to adopt American 
e values which would improve them 


(a) (b) (o (a (e) Change your life style in the direction of 
i the host-country's life style 


(a) (b) (e) (a) (e) Photograph the country and its people 
(a) (b) (c) (d) (e) Purchase objects of art, souvenir itens, etc. 


(a) (b) (c) (day (e) trend host-country ceremonies and festivals 
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43.— Estimate how often you have social or personal contacts with host- 
country nationals in the situations listed below. J. 


51. 


43. 


44. 


45. 
46. 
47, 
48. 


49. 
50. 


51. 


(a) 
(a) 
(a) 
(a) 
(a) 


(a). 


(a) 


(a) 


(a) 


(b) (c) 
(b) (e) 
(b) (c) 
(b) (o 
(b) (c) 
(b) (o 
(b) (o 
(b) (o 
(b) « (c) 


(d) 
(d) 
(d) 


: (d) 


(d) 


(d) 


(d) 





So. Less than once a month. 


(e) 


(e) 


(e) 


- Off-hours association, with enlisted personnel 


uc 

a. Several times daily , 
.b. About once a day | 
c About once a week 

d About once a month 


Oa-the-job association, with enlisted personnel 


On-the-job association, with officers ` | 


Off-hours association, with officers , 
Commercial interaction with merchants 
Casual interaction with civilians 


Discussing personally revealing topics with 
a host national 





Recreational activities (i.e., parties, games, 
sports, etc.) 


Eating a m with a host-country family 





Sta 
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52.- During your training experience did you have any difficulty with 
' 65. the. following? 


a. Much difficulty 


. b. .Some difficulty 
l c. No difficulty 
oct Uds No opinion 
c 


52. (a) (b) d) 








Length of training sessions 
53. (a) (b) (c) (a) EE of — work 
54. (a). (b) (c) (a) Communication with the staff 
55. (a) (b) (c) (a)^ Pace of training 
56. (a) (b) (c) (d) Other students in your training class 
57. (a) (b) (c) (a) Number of tere in your class 
58. (a) (b) (c) (d) Amount of material presented (too much) 
59. (a) (b): (o (d) Amount of material presented (too little) . l 
60. (a) (b) (e) (a2 , ` Amount of homework 
61. (a), (b) (c) (a Handouts 
.62. (a) (b) (co (d) Taped presentations e 
63. gen (b) (e) (a) Amount of knowledge displayed by the staff 
64. (a) (b) (c) (a) Being able to present veu upiuiuni 
US ` l e 
65. (a) (b) (c) (d) Getting feedback on your performance' 
. ° 
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